75 research outputs found
Nearly extensive sequential memory lifetime achieved by coupled nonlinear neurons
Many cognitive processes rely on the ability of the brain to hold sequences
of events in short-term memory. Recent studies have revealed that such memory
can be read out from the transient dynamics of a network of neurons. However,
the memory performance of such a network in buffering past information has only
been rigorously estimated in networks of linear neurons. When signal gain is
kept low, so that neurons operate primarily in the linear part of their
response nonlinearity, the memory lifetime is bounded by the square root of the
network size. In this work, I demonstrate that it is possible to achieve a
memory lifetime almost proportional to the network size, "an extensive memory
lifetime", when the nonlinearity of neurons is appropriately utilized. The
analysis of neural activity revealed that nonlinear dynamics prevented the
accumulation of noise by partially removing noise in each time step. With this
error-correcting mechanism, I demonstrate that a memory lifetime of order
can be achieved.Comment: 21 pages, 5 figures, the manuscript has been accepted for publication
in Neural Computatio
Computational role of sleep in memory reorganization
Sleep is considered to play an essential role in memory reorganization.
Despite its importance, classical theoretical models did not focus on some
sleep characteristics. Here, we review recent theoretical approaches
investigating their roles in learning and discuss the possibility that
non-rapid eye movement (NREM) sleep selectively consolidates memory, and rapid
eye movement (REM) sleep reorganizes the representations of memories. We first
review the possibility that slow waves during NREM sleep contribute to memory
selection by using sequential firing patterns and the existence of up and down
states. Second, we discuss the role of dreaming during REM sleep in developing
neuronal representations. We finally discuss how to develop these points
further, emphasizing the connections to experimental neuroscience and machine
learning.Comment: Accepted for publication in Current Opinion in Neurobiolog
A Hopfield-like model with complementary encodings of memories
We present a Hopfield-like autoassociative network for memories representing
examples of concepts. Each memory is encoded by two activity patterns with
complementary properties. The first is dense and correlated across examples
within concepts, and the second is sparse and exhibits no correlation among
examples. The network stores each memory as a linear combination of its
encodings. During retrieval, the network recovers sparse or dense patterns with
a high or low activity threshold, respectively. As more memories are stored,
the dense representation at low threshold shifts from examples to concepts,
which are learned from accumulating common example features. Meanwhile, the
sparse representation at high threshold maintains distinctions between examples
due to the high capacity of sparse, decorrelated patterns. Thus, a single
network can retrieve memories at both example and concept scales and perform
heteroassociation between them. We obtain our results by deriving macroscopic
mean-field equations that yield capacity formulas for sparse examples, dense
examples, and dense concepts. We also perform network simulations that verify
our theoretical results and explicitly demonstrate the capabilities of the
network.Comment: 34 pages including 21 pages of appendices, 9 figure
- …